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r The Comprehensive Tests of Basic.Skillsf (CTBS/S) was 
the nationan.y standardized, norm-referenced test selected for the 
Idaho Statewi% Testing Program for 1976-77. It provided basic skills 
measures in reiitling/ language, mathematics, total battery, reference 
skills, science and social studies, Studefnts in .the \4th,^ 8th and 11th 
grades from 51 of ^4aho»s 115 school districts were selected on a 
random basis for participation in the testing program. The sampling 
procedure provided a sample representative of the state's students 
both by geographic ^Location and by sizeybf schT)ol attended. Analysis 
of the data compared the mean (averag^y jjerf ormance of Idaho students 
on the CTfiS/S measui^es to the mean performance of the national ^rm , 
group, and compared Idaho's upper and students to their \ 

counterparts in the national norm group, wt of the /analysis came ' 
thTee major findings^ (1) The^ mean performa^K^e of Idaho's students at 
the 4th, 8th .and^ 11 th grades was either equalHo^or better than the 
national norm on each of. the 19 CTBS/S total scores for reading, 
language^ mathematics, battery, reference skills, science and social 
studies. (2) On 41 of the 43 CTBS/S scores reported at the three 
grade levels, the mean performance of Idaho's students i^as either 
equal to or better than the national norm. The two exceptions were 
4th grade spelling and 4th grade mathematics computation. (3) Idaho's 
lower achieving stuA^ents in the 4th, 8th and 11th grades iironsistently 
petfori^ed better on the CTBS/S than their national counterpart, with 
the exception, of 4th gr^de spelling and 4th grade mathematics 
computation. The relationship ,6f Idaho's upper achieving- students to 
their national counterpart varied f^oa test^to test and across the* 
three grade /levels. (BC) 



* Documents acquired by EBIG i*^lude many informal unpublished 

* materials not available from other sources. EHIC makes every effort 
to obtain .-the best ^copy available. Nevertheless, items of marginal 
reproducibility e^re often encountered and this affects the quality 
jf the microfiche and hardcopy reproductions ER^C ' makes available 

* via the ERIC Document Reproduction Service (EDRS). EDRS'is not 

* responsible for the quality of the original document. Reproductioi^s 

* supplied by/EDRS are the best that can be made from the "original. 
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FOREWORD 



You have in your hands the results of the Idaho Statewide 
Testing Program for 1976-77. the program. used a standardized - 
test or basic skills achievement to measure the educational 
progress' of Idaho students at grades 4, 8 and 11 



Special sampling techniques were used to make sUre the . 
program provided an accurate representation of how. Idaho stu- " 
dents are doing, and they're doing well. Tm pleased with the 
results. / 

Jhey point to some areas that need t6 be strengthened, but 
in most xases Idaho students are months ahead of their national 
couhtefparts. V f . 

But most important, the testing program provides us some 
distinct Idaho benchmarks to measure ourselves against in the 
future. In the statewide administration of the* test, prepau- 
tions were taken to ensure the accuracy of the "results." It was 
administered aver a 10-day period and under comparable condi- 
tions by specially trained people. • 

Although there is always some margin for testing error, we 
have confidence in the -program and faith that these Results do 
accurately represent the performance of Idaho students. 

This first set of Idaho norms^should enhance faith in pub- 
lic education in this state, but, more important, itralso sets 
some high marks to maintain and exceed in the. future. 




Roy Truby ^ 
State Superintendent 
of Public Instruction 
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IDAHO STATEWIDE TESTING PROGRAM FOR 1976-77 . v - 

SUMMARY, ' • . 

The Idaho Sta-tewide- Testing Program for 1976-77 wa^ con-r 
ducted th||^equest of thf. Idaho Senate's Health, Education" 
and Welfare ConimitteQ. Testing materials and comiJuter scoring 
services were purchased with federal funds available on a one- 
time basis only^ while other- expenses were absorbed' by the 
State Department of Education and the local school districts 
participating in the project. ' • » 

The Comprehensive Tests of Basic Skills (CTBS/S) was the. 
nationally' standardized, norm-ref erenaed test selected for 
i^Jite program. It provided basic skills i^easures in reading, 
language, mathematics, total battery, reference skill-s , science 
and social studies. 

Students in the 4th, 8th and 11th grades from 51 of ^ ^ 
Idaho's lis schpol districts were selet:ted on a random basis 

for par^ticipation in the testing progr^am. The* sampling pro-, 

\ ■ - . ' ■ ' ' 

cedure prpvided a sample representative of the state's stu- 

dents both by geographic location and' byx^ize of school 

attended. The sample included 977 4th .grade students from 

56 schools; 995 8th grade students from "25 schools and ,997 

11th grade students from 13 'schools.* 

* Analysis of the data 1) compared the mean ' (average) per- 

formance of^ Idaho students on the CTBS/S measures to the mean 

performance of the national norm group, and 2)^comp^red< 

Idaho's upper and lower students to their counterparts in the 



national norm group • Out of the analysis came three major 
findings: " 

(1) The mean perfdgcmanc'e of Idaho's students at the' 
4th, 8th and 11th grades was either equal to or better than 
the national norm on each of the 19 CTBS/S total scores foif^ 
reading, language, mathematics, battery,, reference skills, 
science and social stu<3ki.es,^ 

(2) On 41 of 'the 43 CTBS/S scores refporte^d at the' three 
grade levels, the mean performance of Idaho'' s students was' 
either equal to or better than the national norm,- 'The two, 
exceptions were 4th grade spelling and 4th gth^^e jraathematics 
computation. , . . ^ 

(3) Idaho's lower achieving students in the 4th, 8th 
a'hd 11th' grades consistently performed better on the CTBS/S 
thajv their national counterpart, with the excepJ:ion jjjj^ Ath 
gra^f spelling and.,^4th' gradq mathematics *computat ion. The 
relationship 'of Idaho's upper aphieving students to their> 

national counterpart varired from test td test arid across th.e 

V _ • 

three grade levels. % - , = 



A- 



INTRODUCTION 



[•his report describes the procedures and results of th^ 
*Idaho Statewide Testing Program for 1976-77. The statewide 
testing program was put together in response to c:a";request ^ 
from the Idaho Senate's Health, Education and Welfare ^EW) 
Committee. The Committed asked the State Superintendent to 
"conduct a random ^sampling of school districts to try to / 
.find oiit what is now being done in terms- of th^isic skills." 
The statewide testing program measures where Idaho students 
stand according to a widely used national standardized-' test , 
the Comprehensive Tests of Basic Skills. The purp6se of this 



assessment of achievement is ultimately ta\mprove educational 

• 

practices for the benefit of Idaho students. 

Any valid statewide testing program m^ist have results 
based on a sample which is truly representative of all Idaho 
Students. This was the f irSt concern which had to be met in 
designing the testing program. . \ 

Since the Idaho Legislature.idid not appropriate any money 
for implementation of the program, financial constraints re- 
quired the statewide testing program to be limited to a sample 
of students from three grades. Grades 4,^8 and 11 were se- 
lected because they represented the "middle grades" of^^each 
of the three public school levels — elementary,, junior high 
arra senior high. In this way, a survey of basic skills 
achievement adross the entire 12 years of- pjublic school* en- 
deavor was provided. Furthermore, the random selection of 
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students at.*each gradef le\iel provided virtuallW every student 
in th^ state (in grades 4, 8 and 11) a chance , to be selected 
as part of the ""testing program. ' Thus the ..Statewide testiAg ^ 
Program fdH 197 6-J7 is protected from the prin'cipal objection 
.encountered Jpy two earlier '^efforts to determine' th,e s'ta^s cbf 
basic skills am^ng Idaho stuflents, the-orbjection that results 
were not ^complete and/or not Representative of all Idaho stu- 
dents-. ' . / * * ^ " ' * 

History of Statewide Assessments r ' 

* ^ In 1959 / a statewide assessment, project based on the 

Iowa Test of Educational .Development' (ITEj)) was, impr^weni^ed 

.at grade 11 with funding from the National Defense Education 

Act -until 1965. Then money provided bir th*e Elementary and , 

Secondary Educ/atidn Act ^paid for it until 1975; That yea,r 

- , . ^ ' ■' / < 

the project terminated when options on Jiow to spend feder^al 

funds earmarked" for testing" vi^ere shifted f^rom the state to 

the school districts, leaving the state without^ means to'con- 

timie the program. Prior to 1975 most of Idaho's school 

districts took advantage of the testing program, even tftough 

participation was -voluntary * RepCrts ii?cluded standard in- 

dividual .and group information as well as Idaho percentiles 

against which each district could, rat^ itself. However,.' 

the fact that the p:^j^ct was limited-^to grade^ll made it 

difficult to ascertain what was happening ^statewide in achieve 

mi&nt before students became juniors. 



■ In 1975, a special study committee, the Superintendent's 
Commission on B^s4.c Skills, reviewed test results of 59 school 
districts. Becausfe it was a "basic skills" commission, lit . 

limited its study to reading and mathematics performance o'^f 

' , ^ \j " 

pupils in grade 3 and grade 6, but it did include data <:over-, 
ing about 13,000 students (about' half the student populations 
at those. grade xuyels) . 

The major difficulty the commission confronted was that' 
tlfe data, which had been generated through local actlfvity 

if 

\afther than a statewide effort, came from four different 
i^andardized tests, each with :yts own set of norms. Nonethe- 
Aess, the comijiission concluded from its analysis tT^t a 
(greater proportion than it expected of .Idaho's studenN^s were* 
at or above grade equivalent in the areas • reviewed . Thb^e-*^ 
port of the Superintendent's Commission on Basic Skills 
several acknowledged shortcomings. It was recognize* tha^ 
the report covlpred in rough fashion how Idaho students did 
only in one year — it was limited iiy scope and provided no 
real ' benchmark to measure future years against. /I 

^ Program Funding 

Money for the new statewiJd^'g'|G^ing program was available 
only 'for 'the 1976-77 school yea»fe ^ Since no state resources • 
werdl appropriated, federal funds (carryover Fiscal Year 1976 i 
money un^r Title III of ,the Elementary and Secondary . Educa- 
tion Act) which were earmd.rked f^r^guidance, counseling and 
testing were directed tp the program. These funds were suf- 
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ficient only/ to purchase testing materials'^and scoring serv- 
ices; all p^fs^onnel costs of the state Local agencies 



ass 



ociajr^^with the "^^^ogram were absorbed by those agencies. 
Unfortunately, no like federal money will be available in the 
future to fund Idaho's statewide testing program. These funds 
are no longer ' available to us since the Title III program is 

f • ^ : 

being phased out. . * ^ 



Program Design \ 

The design of the Statewide ' Testing Program for 1976-77' 
- - I 

did: 

(1). provide statewide descriptive data about how Idaho 
students achieve in comparison tp a natiohal. norm 
in the performancejbreas of reading, languad'^, 
mathematics, reference skills, sci^tice andj|«cial., 
studies, • ^iy> 



(2) establish! special "Iddho norm talSj^s" based on the 
performance, of students in the Idl^o sample, and 

(3) provide, extensive information to pirticipatinig 
scjiools both in the form of standard commercial 
scoring reports of individual and group - results 
and in the form of the special "Idaho norm tables." 

This information has the potential of aiding teachers and ad- 
ministrators in making instructional and administra-ti^e 4®^^" 
sions. ^ . ^ ' 

' It|s also important to say what the Idaho Statewide Test- 

ft 

ing Program for 1976^77 is not designed to do and should not 
be used for. It did not : 

(1) measure important aspects of education that are 
beyond basic skills achievement, 

<2) provide for any .direct evaluation of specific edu- , 
cational programs or acjiivities, ' 



\ ■ ' ■ * 

(3) provide for the comparison of schools school 
districts to on^i anothe^), 

(4) provide, a way to evaluate teacher effectiveness or 
administrative ['^efficiency, or * 

(5) provide information about spegi/Eic cause(s) of the 
reported achievement levels. 



These provisions are far beyond the possibilities of ^ 



Standardized 
auJ:hority. 




adiainisteired wjft:h\Limited» resources iind , 
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- PROCEDURES 

What follows is a description the methods and mate- 
rially used for the Idaho- Statewide Testing. Program for- 1976- 
77. It^:cbnsist& of four sections: the first described the 
standardized, norm-referenced test used for the program, the 
second presents a view of the state sample and how it was 
sheeted, the thiird discusses the procedures used to admin-^ 
ister the test, and the fourth outlines the data processing 
and analyses methods used. 

The Test 

Expanded Edition of the Comprehensive Tests of Basic 
gkills (CTBS/S) , published by CTB/McGraw-Hill of Monterey, 
Californids, was the standarcjized test used for the Idaho 
Statewide Testing Program for 197 6-77. ' The CTBS/S was .se- 
lected via a competitive bidding procedure among the major 
test publishers. Minimum acceptable requirements for the 
test and for the desired scoring services were specified ai>d 
Ipids were requested. CTB/McGraw-Hill met, or exceeded, every 
specification and did so witl> the lowest of the .three bids 
submitted.. n3 . . 

The CTBS/S provided measures ^f achievCTient in reading, 
language, mathematics, reference skills, science and social 
studies. A list of the 15 separate CTBS/S scores and a brief 
description of what each score -purports to measure is present 
ed in Exhibit 1. The entire battery of tes'ts, with the ex- 
ception of the science and social studies^ tests at grad^ 4, 



. Reading Vocabulary : recall of. synonyms. 

Readi ng Compreherts ion : literal recall, rewording, determining word 
meanings from Context, main idea, descriptive words, conclusions and 
strdcture/style. . -,' 

Reading Total : combination of the two reading tests. 

Language Mechanics : pynctuatlon and capitalization. ^ > 

Language Expression : grammatical forms, diction, sentence structure 
and paragraph organization. 

Spelling : recall of rules, use of homonyms (to, too, two) and \^ 
easily confused words. 

Language Total : combination, of the three language tests. 

Mathematics Computation : adding, subtracting, dividing, multiplying, 
raising to a power and extracting a root. 

Mathematics Concepts & . . 

MathematicT Applications : number systems/properties .geometric 
relationships, measurement, sets, graphs, mathematicM^sentences , 
problem solving and reasoning. (Note: these two tests are given 
as a single test.) y . - ' 

Mathematics Total : combination of the three ;nathemai^ics tests. 

Total Battery : combination of ^he reading, language and mathematics 
tests. ^ . 

Reference Skills : parts of a book, (title page, table of 'contents 
and index), dictionary skills (alphabetization, guide words, 
dividing words into syllables and multiple definitions), and library 
use (catalog cards and reference materials). 

\ 

Science : process skills (knowledge recall, classification, interpre- 
tation of data, hypothesis evaluation, etc.) in chemistry, physits, 
earth science, ecology, botany, zoology and general science. 

- ■ J- 

Social Studies : physical geography, environments, individual and 
• group interactions, social systemi|| pol itica.1 systems, economic 
syste^ns and history. a/ 



EXHIBIT 1.' CONTENT OUTLINE FOR THE COMPREHENSIVE 
TESTS OF BASIC SKILLS (CTBS/S) 
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• . . • • -10 

were adminisLtered to the participants- of , the statewide test- 
ing program. The science and social studiei^ tests were not 
given at grade 4 because of financial constraints at the time 



specif iqations for the -testing program vrare written. 

. ' ' /// 

. An advantageous feature of the CTPS/S is that it offers 

• a uniform reporting format for all students participating in 
the program regardless of grade or /test level. The C-TBS/S 
battery consists of several level's , each of which measures 
progressively^ more difficult and more complex performances 
^within the basic skills areas-1 For the statewide testing 

.Jprogram, grade 4 took Level' 1 of the CTBS/S, grade 8 had 
Level 3 and grade 11 use^/ Level 4. These were t^ appropri- 
ate levels as recommended by the test publisheo^ ) 

In 1973, the CTBS/S was standardized on nationwide 
sample of some 130,060 students from each of tKe 50 states. 
The norming, sample included public and parochial schools, 
various ' types of communities and school districts 6f various 
student densities (enrollment per square mile) . 

* Th^ CTBS/S was normed in the month of April, but Idaho 
students were tested in September. This fact required that 
Idaho students be compared against a statistical projection 
of what the nationwide sample might have achieved had it taken 
the- CTBS/S in September rather than in April. The statisti- 
cal procedure used tp determine the projection, which involves 
a linear interpolation/extrapolation of obtained data, is a 
fairly standard practice often used by test publishers. The 

15 



variance in testing. times, however > adds an. unknown factor to 
the comparison of Idaho versus National results. 

The Sample . *'\.'v ^ ' 

The sample for the Idaho Statewide Testing' P^rog'ram for 
1976^7 included students from grades 4/8 and 11. These 
jrades represent the "middl^ of each of the three divisions 




>ublic educational enterprise:, elementary^, ;j,unior 
high and senior high. The students were selected by a clus- 
tered two-stage sampling procedure. The two stages required 
1) that schools be selected on a , random basis for participa- 
tion in the testing program and all their students be tested, 
and 2) that individual students be selected from all those 
tested on a random basis from the particMiMLng schools. 




Selection of Schools , The selection of schools for the 
1976-77 testing program was completed during the spring of 
the 1975-76 school year. The first step was to identify all 
Idaho schools which offer instruction .in grades 4, 8 and 11, 
and to estimate their probable 1976-77 enrollment. 

All schools in the state thus identified were then class- 
ified according to two criteria: school size and -geographic 
location. Each school was designated as "small," "medium" 
or "large," The actual nyimber of students defining each size 
varied at each grade level. Also, a term indicating geo- . 
graphic location was asisigned to each school (based on the 
regions of the Idaho Association of School Superintendents), 
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■ - ■ ■ ■ ■ - ■ . ' ■ - ■■-./Af- 

The terras were "northern," "we*ea:n" and "eastern." S'cho&ls ..f 

■ ■ ■ //■ / :/ 

of each sort (i.e. small-northern, middle-northern, /la^e- / 
northern 4 small-western, etc.) then were selected/bn ^ ran^t^m 
basis. The procedute,^ which was repeated at each ^4ade.^vel/ 
resulted in the se^ectiiDn of schools from 51 of tlM state's 
115 districts. These included 56 schopls for the 4th grade 
level test, 25 schools for the 8th ^rade, and is schools for 
the. 11th grade test. * * •/ 

■ : ' : ■ •,• ■■ ' ■ ■// ^ ' 

Selection of g^ude^ts . Selection of^ndividual students 
was made .after the testing in the school^ h^d been completed. 
The sampling procedure called for a specific number of s^l'u- 
dents to be selected on a random basis from each school/. 
Only students who had completed the/entire battery we;re con- 
sidered eligible for inclusion in/the state sample, /in total, 
91%^of the estimated student eni^/j!!lment of participatit^ 
schQols produced usable answer sj/eets . ^ 

Participating schools were /instructed to test all stu- 
dents except those whose mentai^/^or physical handicap were of 
such extent that the^ could no^t possibly take the test. Stu- 
dents with specific leatning ^isat^ilities did participate in 
the testing. . / ./ 

The sampling design .cal|Led for the selection of 1,000 
students out of each grade ^rom jthe 3,502 tested in the 4th 
grade, the 3,815 tested in the 8th grade and from tlie 2,736 
tested in the 11th grade, pue ^to a margin of error from* 
various sources > however^ ihe ^election actually resulted in 



^ • . 1 

a sample which included 977 4th grade students, 995 8th grade 
students and 997 11th grade students. ^ ,; , ' ' 




tive of all students in the state. The criterjla for r§pire- 



gentativeness for the Idaho Statewide Testing f^rogram 'f or. 
1976-77 are school size and geographic location. 

Exhibit 2 displays at each grade level the number of 
students in the 'state sample and* the number of Istudents in. 
the estimated state^ enrollment for each size-area cell. (i.e« 
small-northern, large-eastern, etc.)*f For example, in the 
4th grade there are 3^ students in the sample from small- " 
northern schools where there .was an estimat'^d enrollment of 
718 Istuc^ents. . „ ' - 

Exhibit 2 also presents the percentages of students in 

the state sample and in thp estimated state enrollment by 

^ ' ■ : 

school 6ize and by geographic location. For example, in the. 
4th grade 21% of the sample came' from hdxthern schools while 
21% of the estimated state enrollment attended those same 
.northern schools; likewise, 14% of the > 4th grade sample came 
ofrbm small schools — small schools contained 15% of the total, 
'(estimated 4th grade state enrollment . These .percentage fig- 
ures indicate the practical representativeness of the sample. 
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School Size"^^-^ 
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State 
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Smin 1-29 


32 


718 


. 39 


701 


60 
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1' 14% 


,15% 






Medium 30-65 


100. 
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113 
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1 30% 


30% 
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50 ' 
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50 


472 


1 15% 


14% 






Medium 60-200 


50 


1051 


150 


2209' 


' 151 


2031 


1 35% 


33% * 






Large 201+ ' 


102 


1722 


244 


4610 


149 


2369 


5.0% 


53% 
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906 
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• 20% . 


' 22%' , 






Medium 100-250 
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100 
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EXHIBIT 2, FREQUENCY AND PERCENTAGE DISTRIBUTIONS FOR 
GRADES 4, 8 AND 11 BY GEOGRAPHIC LOCATION AND SCHOOL SIZE 
FOR STUDENTSft IN STATEWIDE TESTING SAMPLE AND 
- ESTIMATED "SCHOOL EI^RQLLMENT 
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The sample may be considered representative of the state 
student population in a practical * sense Inj that^ttife sample 
percentages and estimated enrollment percentage^^ are similar , 
both by geographic location and by school size.. However, 
there is no generally accepted criterion indicating how wide a 
difference in percentages can exist before the sample becomes 
"nonrepresentative. " From use of a cr,iterion point of 5% 

(arbitrarily chosen at the' discretion of the author^*, it can 
fbe concliided .that , the 4tb;and 8th grade seass^m^^era indeed ^ 
representative of their respective stat^^^ both by 

gepgraphic locatiqn and school sii^*^^^^ .J^^^^^grade sam- 
pie was repijesentative of state enrdi^]^a:;S by ^school size, 
but is questionable when geographic lot^tion is considdj^ed 

(northern schools were slightly over^^eipresented while stu- 



dents from eastern schools were unde'r-represented) . However, 
this variance is not considered sufficient enough td . seriously 



distort the statewide results of the test for the 11th grade. 

Administration of the .Test 

Th^ Comprehensive Test of » Basic Skills (CTBS/S) were ad- 
ministered to most schools in the statewide testing program 
on Septenfcer 28-30, 1976. Schools in three eastern Idaho 
districts were allowed to test one week earlier because an 
early fixost had moved their harvest vacations into the last 
week of September;.- -Although ^standard testing procedures 
jfcecommend that at least^ two full school days be aldowed for • 
the completion of the CTBS/S battery, the timeline for the 
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state program required testing time to be limited to one and 
one-half school days in. the^4l;h grade and one day (or two 
Vhalf-days) in the 8th and 11th grades, ThiSTeft'^an unreal- 
istic one to one-and-a-half days for make-up testing, The# 
statewide testing effort was complleted basically as sched- ' 

uled/ except for one school which did not administer any 

■ ■ ^ 

mathematics tests to its '4th grade sj:udents (this was not - 
"discovered until after . the answer^heets- had been scored). 
The extent of make-up testing is not known. \ ' \ 

The administrajbion o^ the CTBS/S battery- in each school 
'was supervised by a "test coordinator" appoirited b^ the dis- 
trict superintendent and trained by State Department of Edu- 
cation staff. It was the option of the tist -coordinator to 



he T)c 



determine who would actually adm^jiisbS'r the ibattefy to the 
^students as each sc^hoolJiand each, grade level had it^ own 
unique situation. Generally , teachers administered the ^est 
in the elemenmry schools and counselors gave the tests in__ 
the ^^^coridary schools.-- In on^gmall district, howfever, the 
superintendent himself admdniste'red the CTBS/S to his 4th 
grad!fe students. The test coordinator was expected to either 
give the tests himself /herself or instruct those serving 
as examiners about t^tie standardized administ^^ration proced- ^ 
uires. 

During the first two weeks of .September, workshops foir 
test coordinators were conducted by the State D^artment of 
Education in six locations around the state — Idaho Falls, 
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Pocatello, Twin Falls/ Boise, Lewiston and Coeur d'Alenfe. 
Each vrotkihpp, which lasted approximately four hours, was 
designed to acquaint test coordinators wiSthf the history -and • 
development of the goals and objectives of l/the testing* pro- 
gram, to review the CTBS/S materials and standardized pro- 
cedures to be followed in the administration, to revfew J'^ 
prepafation of answer sheets for scoring, to consider , the- 
format and us^ of the extensive information the participa- . 
ting schools would receive . about their students, and to 
answer any: questions about the program. - 

The 'CTBS/S Jaaterials, which included the test battery 
and answer sheets, were mailed directly to each participating 
school by the test publisher. Since the quantities sent to 
each school were ba^d on an enrollment' estimated from the 
previous school year^ there was some confusion before all 
schools had received all the materials they needed. After 
the testing was completed, the CTBS/S materials became the 
property of the schools. 

The state/sample upon which the results |of the statewide 
testing were based was smaller than the 1,00C[ students at 
each grade level which had been planned. The %dif ferencg can 
be'^countgd for, generally', by two factors. 

^^irst, two schools tested fewer students than had been 
projected. One school ha4 a smaller enrollment than estimated, 
SO after al^ its eligible students were included in the state 
sample, .it was still short of its "allotment. The other. 



sch^ool dlan't conform to the st^te prograrri requirements in ^ 

that it did not administer math tests .to any of Its^ students, 
% • 

thus making the whole school ^Meligible for inclusion, in the 

x" / - ^ ■ 

State saijiples. . * 

Secoi^, eath test coordinator was charged with the 
responsibility of p'reparing the answer sheets from his/her 
school for selection /of 'tAie state sample, by the State Depart- 
ment "of Education an^ for scoring by the publisher. This 
preparation consisted of dividing the answer sheets into two 
groups-^one of alphabetized complr^ed answer sheets, 'i.e. 
answer sheets which indicajbed that th^ student had participated 
on each of the CTBS/S tests administered, and one of incom- 
plete answer sheets. The answer sheets of all students ^ 
would be scored and the results reported to the schools, but 
only, completed answer sheets were eligible for the sjiate 

>le. This assignment, however, was not accurately com- 
jl^leied by all test coordinators, resulting jp.n the selection 
of ^i(OTie incomplete answer sheets for the state sample. It 
was decided Jfcfiat, as far as possible, the incomplete answer 
' sheets would be; deleted froi!^ the state sample. 

Data Processing and Analyses \ 

After testing was completed in the schools, all answer 
sheets were sent to the State Department of Education.. There 
the answer sheets of students in the state sample were marked 
for future cortiputer,. . identification. The materials were then 
'shipped to the test publisher for computer scoring. School 
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results were mailed directly to the test coordinator for each' . 

srSltbol. The only information ^i^urned to the State Depart- 

/ment of- Education was summary reports on the studisnts in the 

state sample v / The data collectipn anci^rocessing. procedures 

were designee! so that no individual student, t|^cher, school 

or disttict could be identified by the information . housed 

within the State Department of Education. 

■ ' ' ' ^ \i 

The scpring reports sent directly to the schools included 

the following: ' ^ f ' • 

(1) Individual Test Record. This report shows the 
student's performance on each of the CTBS/S tests . 
in terms of four scores (raw score, scale score, 
national percentile and Idaho percentile) .It 
also shows which questions the student attempted 
and which of those the student answered correctly 
or incorrectly. 

(2) Self-Stick Label. This label provides a conven- 
lent way of recording the four student scores on ^ 
each CTBS/S test in the student's folder. 

(3) Clas^ Record Sheet . This report lists in' alpha- 
^ betiftil order each student in- a classroom and . , 

displays their four scores. 

(4) Right Response Summary, T^is report shows for 
each question on the CTBS/S tests the percentage 

of students in the classroom who answered • correct- . 
ly' and" the perceifitage of students in the national 
' norm group who answered correctly. 

(5) Frequency Distribution Chart. This report pro«- 
" -vides for each school a sximmary description of 

the distribution of its students over the rangfe 
of grade equivalents and the mean grade equiva- 
lent on each test for the school as a whole. 

This info'rmation was provided to each school as .a "reward" 

for its participation in the* statewide testing program. Each 

school wa,s informed that the analysis, interpretatioiv and. 
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' public distribution of its results should be conducted accord- V 
■ ^ ^. 
ing to district guidelines and policies. 

>' • _ _ . ■ , ' 

^ / The scoring reports received at the state level included 

the Allowing ](xine seizor each o^"" the three grade levels): 



(1) ThWe State fi^quency distrubitions (one each for 
■ raw ..scores, gj/a^e equivalents and scale scores) 

which .describes the distribution, range and mean 
of students in the state sample for each of the 
^TBS/S tests. . ^ 

(2) The rxght response sumlliary ^hich' lists, for each 
question on the CTBS/s: battery V the percentage- of 
students in tl\e state sample^ wbo answered correct- 
ly and the percentage of -national norm students 
who answered' correctly. 

^ (3) A cluster analysis /showing descriptive statistics 
^or each cluster, .and a customized statistical 
analysis which establishes the 95% Confidence 
> , Interval for the mean scale score on. each CTBS/S ; 

^ test. : . . ^ " 

The results reported in this paper primarily come out of the ^ 

state frequency distributions with use of some right response 

■ ■ ^ < .. -■ 

^uiranary information to. examine some of the findings in more 
detail.. The outcomes of . the fluster analysis .and the custom 
statistical analysis will not be reported in this paper^ The 
usefulness "of their resultant information depends on a. strict 
adherence to statistical and procedural requirements.^ Unfor- 
tunately, there were enough irregularities during the operation 
of the. statewide testing program that the results of these two 
analyses are not subject to statistically .valid interpretation. 

The analysis of data gathered' through the statewide test- 
ing program was directed to three general questions: 
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(1) How do the thr^l^grades taker] /together compare to 
the national norm 'in mean performance on' each of 
±he rSajor subject arteas measured^ by the CTBS/S?, 

(2) How did each grade compare to the ttatioaal norm on 
mean performance on each of the CSBS/S' tests? 

(3) How did Idaho's "lower" and "upper" students compare' 
to their counterparts in the national norm groups? 

The answers to these three questions are, discussed in the next. 

' ■ . ' ' . ^ t( ■ • . . ^ •. • 

chapter. 
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RESULTS 

. . • -I 

This chapter presents in four sections the results of 
the Idaho Statewide Testing Program for 1976-77. The first 
section defines the scores used in data analysis and report- 
irig. The second section describes mean (average) student 
achievement for each of seven CTBS/S subject areas. The 
third section presents mean student achievement on the CTBS/S 
for each of the three grade levels. And last, the fourth 
section displays the basic skills achievement of Idaho's 
lower and upper students. 

Definition of Sc ores 

The results of the statewide testing program were com- 

'\ 

puted and; are reported in terms of several scores : . 

(JL) Raw Score. When a ±est is scored, the first score 
obtained is the raw score or, simply, £he number of 
questions which were answered correctly. The raw 
score lacks meaning because it cannot be compared 
to a raw score on other tests. This is because the 
/ number of items ^nd the difficulty of items differ 

from test to test. 

(2) Scale Scox^. Scale scores are produced from a 

single^ equal interval scale of scores across all 
grades for use with. all levels of the CTBS/S. 
These scale scores, expressed i^ three-digit numbers 
from 000 to 999, enable the user to chart a student's 
growth from kindergarten through grade 12, rega^rdless 
of which CTBS/S levels were given.. Scale scores 
have "^the statistical advantage over other scores 
(such as grade equivalent and percentile rank) 
in that they can be averaged. Scale scores, 
however, do not have intrinsic meaning for users 
unless the users have a thorough grasp of the 
concept of the normal distribution curve. » 
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(3) Grade Equivalent, The grade equivalent for a par- 
ticular scale score represents the :j^ar and month 
of school for which that 'scale scolre is the median. 
For example, if the median scale score for all 
students in the aecorid month of >garade 4 (i.e. 4.2) 
is 397 on a certain test, then it is said that the 
grade equivalent of a scale "s6ore of 397 on that^ 
test is 4.2 (4th grade, 2nd month). " 

(4) Percentile Rank. The percentile rank is not the 
percentage ^of items answered correctly. THe per- 
centile rank for a particular score may be inter- 
preted as the percentage' of students in the group 

.0 which had a lower score. It is computed as 

percentage of students with a lower s^ote plu^ one- 
half the number of students with the Scune score* 
Thus, if a partidtilar score has a percentile rank 
of 57, this would mean that approximately 57% of 
the students scored below that score. 

Subject Area Results i 

The CTBS/S provided measures of basic skills in reading, 
language, mathematics, total battery (combination of reading, 
language and mathematics) and reference skills in grades 4, 8 
and 11. Measures for science and social studies were also 
obtained for grades 8 and 11. The results of the three grades 
for each subject area are presented on Figure 1 through Figure 
7: Reading Total (Figure. 1) , Language Total (Pigiire ' 2) , mathe- 
matics Total (Figure 3), Total Battery (Figure 4), Reference 
Skills (Figure 5), Science' (Figure - 6) and Social Studies 
(Figure 7) . * * 

These figures graphically compare the grade equivalent Qf 
^^aho students* obtained mean scale score to the nationally 
expected grade equivalent. Since students were tested in 
September, the expected grade equivalents were "4.0, 8.0 and 
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, Idaho Nat l 



Idaho Nat'l 



Idaho Natl 



Grade 4 Grade 8 Grade 11 

FIGURE 1. IDAHO VS. NATIONAL RESULTS ON THE 
COMPREHENSIVE TESTS OF BASIC SKILLS (CTBS/S) 
READING TOTAL FOR GRADES 4, 8 AND 11 



25 




Idaho Nst'l 



Grade 4 



Idaho Natl 



Grade 8 



Idaho Nat t 

Grade 11 



FIGURE 2. IDAHO VS. NATIONAL RESULTS ON THE 
COMPREHENSIVE TESTS OF BASIC SKILLS (CTBS/S) 
LANGUAGE TOTAL FOR GRADES 4, 8 AND 11 
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Idaho Nat'f! 

Grade 4 



Idaho Nat 'I 

Grade 8 



Idaho Nat 'I 

Grade t1 



FIGURE 3. IDAHO VS. NATIONAL RESULTS ON THE 
COMPREHENSIVE TESTS OF BASIC SKILLS (CTBS/S) 
MATHEMATICS TOTAL FOR GRADES 4, 8 AND 11 
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Grade 4 



Idaho Natl 



Grade 8 



Idaho Nat I 

Grade 11 



FIGURE 4. IDAHO VS. NATIONAL RESULTS ON THE 
COMPREHENSIVE TESTS OF BASIC SKILLS (CTPS/S) 
TOTAL BATTERY FOR GRADES 4, 8 AND 11 
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■ j> Idaho Nat'l Idaho Nat 'I Idaho Nat 'I 

Grade 4 Grade 8 Grade 11 

FIGURE 5. IDAHO VS. NATIONAL RESULTS ON BHE 
COMPREHENSIVE TESTS OF BASIC SKILLS (CTBS/S) 
REFERENCE SKILLS FOR GRADES 4 8 AND 11 



Maho Nat' I 
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Idaho Nat'l 

Grade 11 



FIGURE 6. IDAHO VS. NATIONAL RESULTS ON THE 
COMPRfeHENSlVE TESTS OF BASIC SKILLS (CTBS/S) 
SCIENCE FOR GRADES ^ AND IV 




'Idaho Nat' I Idaho Nat' I 



Grade 8 Grade 11 



FIGURE 7; IDAHO VS. NATIONAL RESULTS ON THE 
COMPREHENSIVE TESTS OF BASIC SKILLS (CTBS/S) 
SOCIAL STUDIES FOR GRADES 8 AND 11 
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and 11.0 for the'^th, 8th and 11th grades, * respectively • 
(4,0 = 4 years, 0 months) '^4 

The results show that for each of the major subject areas 
measured the CTBS/S, the mean performance of Idaho students 
is either equal to (as in one instance) or superior to » (as in 
18 instances) the national norm group. The sii^gle instance 
where Idaho results, were equal to the national results was in 
the 4tli^ grade mathematics total. The most superior perform- 
ance was displayed in 11th grade science where Idaho students 
had k mean performance of 2.3 grade equivalents (i.e. 2 years 
and 3 months) above the national norm group. The comparisons 
for each of the major CTBS/S subject areas follow: 

Reading Total. Figure 1 indicates that on reading 
total measures the mean performance of the 4th, 8th 
and 11th grades were, respectively, *6, 1.2 and 1.4 
grade equivalents (i.e. 6 months, 1 year 2 months, 
and' 1 year 4 months) above what was expected. 

Language Total . Figure 2 indicates that on language ' 
•total measures the mean performance of the 4th, 8th 
" and 11th grades were , respectively , .3, 1.1 and .9 
grade equivalents (or 3 months, 1 year 1 month, and 
9 months) above the national norm. 

Mathematics Total . Figure 3 Indicates that on mathe- 
matics/ total measures the mean performance of the 4th 
grad^was the same as the national norm and that the 
mean performance of the 8^:h and 11th grades were, 
respectively, 1.1 and .9 grade equivalents cibove what 
was expected. 

Total Battery. Figur¥ 4 "TndTc^^ 

measures the mean performance of the 4th, 8th and 11th 
grades were, respectively, .2, .7 and .8 grade equiva- 
lents above the national norm. 

Reference Skills. Figure 5 indicates that on reference 
skills measures the mean performance of the 4th, 8th and 
11th grades were^ respectively , .3, 1.0 anci .7 grade 
equivalents above what was expected. 



36 



32 



Science; Figure 6 indicates that on science measures 
the mean performance of the 8th and 11th grades were^ 
respectively,- 1.7 and 2.3 grade equivallents abqve the 
national norm'. 

Social Studies. Figure 7 indicates that on social 
studies measures the mean performance bf the 8th and 
11th grades were, respectively, 1.2 and 1.5 grade 
equivalents above what was expected* 



Grade Le vel Results 
— T *~ 

Thirteen CTBS/S measures of basic skills were obtained 
fpr the 4th grade, and fifteen CTBS/S scores were obtained . 
for the 8th and 11th grades. The resialts the 4th grade 

are shown on Figure 8, the 8 th grade on Figure 9 and the 1,1th ^ 
^grade on Figure 10. The expected performance, based on 
n^ional norms, is indicated on the figures by a horizontal 
line across the chart with the grade equivalent values of 4.0, 
8.0 and 11 4O for the 4th, 8th and 11th grades, respectively. 
Any deviation from expected performance in favor of Idaho 
students is represented on the figures by a bar extending 
upward from the horizontal line while any difference unfavorabl 
to Idaho students is indicated by a bar extending downward. 

Grade 4. Figure 8 indicates that Idaho's 4tl^ grade stu- 
dents scored at or above the national norm on 11 of* the 13 
CTBS/S measures of basic- skills . The 4th graders/ ^l|fest per- 
formance in relationship to the norm group was on the language 



expression test where Idaho students scored a grade equivalent 
of 4.8 rather than the expected 4.0. Reading vocabulary,, 
reading comprehension and mathematics concepts followed closely 
behind with obtained grade equivalents of 4.7, 4.7 and 4.6, . 




FIGURE 8MDAH0 tS; NATIONAL <REmTS 0^ 
COMPREHENSIVE TESTS OF BASl|^KILLS (CTBS/8) FOR GRADE 4 " 



I 

3 

a 

V 

10 




FIGURE 9. IDAHO V& NATIONAL RESULTS ON THE 
COMPREHENSIVE TESTS OF BASIC SKILLS (CTBS/S) FOR ORADE 8 
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. FIGURE ia IDAHO VS. NATIONAL RESULTS ON THE 
COMPREHENSIVE TESTS OF BASIC SKILLS (CTBS/S) FOR GRADE 11 " 
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respectively. Figure 8 also indicates that Idaho's 4th grade 
students scored below the national norm by two months on the 
gelling test and by three months on the mathematics computa- 
tion test. 

When Idaho students- were matched against the norm group* 
in terms of percentage of correct responses on the spelling 
test, there was no percentage difference in the ability. of 
the two groups to recognize words which had been spelled 
correctly or incorrectly. Idaho students, however, were one 
percentage point below the national norm (64% to 65%)T;When the 
^ ability to recognize "spelling errors" was tested. .However, 
these were not actual spelling errors but rather incorrect v 
uses of homonyms (e.g. to, too, two) . " - ^ 

When Idaho students were comparted with the norm group in 
terms. of percentage of correct responses on the mathematics 
computation test, Idaho students had a one percentage point 
advantage over the national norm on both addition skills (78% 
to 77%) and subtraction skills (73^ to 70%) . However, the 
Idaho ^students were five percentage points below the national 
norm in multiplication skills (65% to 70%) and eight percentage 
points behind the national nora in division skills (60% to 
68%) . 

Grade 8. Figure 9 indicates that Idaho's 8th grade stu- 
dents scored above the national norm on all 15 of the CTBS/S 
measures of basic skiMs. Language expreiSsion and science 
were the two areas where Idaho students excelled most in 
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relationship to the expected grade equivalent of 8.0. They 

scored a grade equivalent of 9.8 in languagiB expression and 

of 9.7 in science. The 8th grade's smallest advajn|.tage over 

the national group was on the mathematics computation test, 

t . . . ' 

where Idaho students scored a grade equive|ileiit of 8.3 or 

thr-ee months above expectation. 

Grade 11. Figure 10 indicates that Idaho's 1.1th grade 

students scored above ttie' national norm ^ on all 15 of the 

CTBS/S measures of basic skills.. Science was the area where 

Idaho students excelled most in relationship to the expected 

grade equivalent of 11.0, by scoring a grade equivalent of 

13.3. The 11th grade's smallest advantage over the national 

group was on the spelling test, where Idaho students scored 

four months al>6ve the norm with a gra^e equivalent of 11.4 • 

Quartile Results . 

Quartile scores provide a way to compare the lower quar- 
ter and the upper quarter of students in the state saimple 
with their counterparts in the national standardization group 
on each of the CTBS/S measures. The Idaho vs. national quar-- 
tile results for thp 4th, 8th and 11th grades are presented 
on Figure 11, Figure 12 and Figure 13, respectively. The 
three .vertical lines on each figure indicate the three national 
quartiles with national percentile ranks of 25, 50 and 75, and 
the three plotted values for each measure represent the Idaho 
quartiles expressed as national percentile ranks. 
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FIGURE 11. IDAHO VS. NATIONAL QUARf ILE RESULfS ON THE , 
COMPREHENSIVE TESTS OF BASIC SKILLS(CTBS/S) FOR GRADE 4 » 
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FIGURE 12. IDAHO VS. NATIONAL QUARTILE RESULTS ON THE 
COMPREHENSIVE TESTS OF BASIC SKILLS (CTBS/S) FOR GRADE 8 
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FIGURE 13. IDAHO VS. NATIONAL .QUARTI1.E RESUL1S ON THE 
COMPREHENSIVE TESTS OF BASIC SKILLS (CTBS/S) FOR GRADE11 
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To read the figures, using Jbhe 4th grade reference skills 
test as an example (see bottom line of Figure 11) , note that 
the lower quartile of Idaho students' scored\at the 3lft 
national percentile rank which is six (+6) comparison potlits 
above the score for the lower quartile of national students • 
Note also that the Idaho upper quartile of students scored/at^ 
the 72nd; national percentile rank which is three (-3) points 
below the national upper quartile of students.^ From this 
example, it could be concluded that in 4th grade reference 
skills Idaho's lower students performed better than expected 
while Idaho's upper students did not. do as well as expected, 
the national norm group defining what is expected. 

Grade 4. Figure 11 indicates that Idaho ^s 4th gra^e? 
lower quartile students performed better than their national 
counterpart on 11 of the 13 CTBS/S measures. Their best per- 
formance relative to the norm group was in reading vocabulary 
where they scored 14 points (i.e. percentile ranks) above the 
national Expectation, ^e two tests where first quartile stu- 
dents fell below the national norm were spelling (t3 p6ints) 
and mathematics ^computation (-6 points) . 

Figure 11 also indicates that Idaho's 4th grade upper ^ 
quartile students performed better than their national counter 
'^part on five of the 13 CTBS/S measures, performed the same' as^ 
the norm group on two measures, and ""performed below expecta- 
tion on six measures. The range of comparison scores was 
defined by the reading comprehension score (+8 points) and 
the mathematics computation test (-18 points) . 



These quartile results for the 4th grade preseht in- 
•formation on the two ^measures — spelling and mathematics compu-. 
tation — where the Idaho students' grade equivalents of the 
mean scale'^ scores we^e below the national noann; It can be 
seen on Figure 11 that on the spelling test both the lower 
(-3 points) ^nd ^upper (-5 points) groups of students scored 
below the national norm. Similarly^ both the lower (-6 points) 
and the upper (-18 points) grdups did not perform as well as 
expected on the mathematics computation test. The difference^ 
however, was that both upper and' lower students contributed 
about the same to the low mean score 'on the spelling test, 
while on the mathematics computation test the -shortcomings of 
the upper group exerted a greater influence on producing the 
below average score than did "those of the lower group. 

Grade 8. Figure 12- also indicates that Idaho ' s . lower 
quartile students performed better than their national counter- 
part on ail 15 of the CTBS/S measures/ . The range of Comparison 
scores was fif<i^m +17 points for reading total and language 
expression to +4 points for language mechanics. 

Figure 12 also\indicates that Idaho's upper quartile stu- 
dents scored better than their national counterpart on eight 
CTBS/S measures, scored the same as the norm group on one , 
measure, and scored below expectation on six measures. The 
range of comparison scores was from +5 points for reading 
comprehension and science to -5. 'points for language mechanics 
and mathematics total. 
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Grade 11, Figure 13 indicates that^Idaho's lower quar- 
tile students perfprmed better than their national counterpart 
onfall 15 of the CTBS/S measures. The range of comparison / 
scores was from +19 points for science to +1 point for refer- 
ence skills. 

.Figure 13 also indicates that Idaho's upper quartile stu- 
dents scored better than thei^ national counterpart On te;i of 
the 15 CTBS/S measures, scored the same as the norm group on 
two measures r and scored below expectation on three measures. 
The range of comparison scores waa from +8 points .for science 

to -4 points l£or reference skills. 

r 
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